Image processing apparatus, mobile terminal device and image processing computer readable program

ABSTRACT

An image processing apparatus includes a gradation quantization processing in a case of performing a region dividing processing in the animation of a moving picture and in an object extraction, in which frequency histograms are created for each frame, a judgement is performed whether a quantization reference value is re-calculated by comparing the frequency histograms, in the case of performing the re-calculation, and a quantization reference value is successively obtained in accordance with the product of the frequency histograms and the error coefficient, thereby realizing a high-speed high-quality image process.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatusfor processing a moving picture and a program for causing a computer toexecute an image processing method and in particular, to a method forimage gradation quantization and animation, region division processing,and object extraction processing.

[0003] 2. Description of the Related Art

[0004] An image processing technique is used in various systems such asa monitoring system and a vehicle automatic operation system. Amongthem, the gradation quantization processing is a processing for reducingthe number of levels of image component value associated with imagecolor information (for example, pixel values of components of RGB andcomponents of YUV) and reduce the number of colors in an image. Forexample, a full color image in which each of RGB is expressed in 256gradations is reduced to about 10 gradations for each color. Thegradation quantization processing is performed as a pre-stage of regiondividing processing for dividing a region into regions each havingsimilar colors in an object extraction processing for isolating aperson, a vehicle, or the like from a picked up image.

[0005] Moreover, a digital camera and a video camera are widely spreadin people. A user performs image processing for processing/editing animage picked up by a PC. It is expected to develop an animationtechnique for transforming a picked-up moving picture into an animationcell image and a technique for attaching a character, a symbol, and apicture onto the image or cut off a particular portion from an image andcombine it with another image. In the moving picture animationtechnique, smoothing and contour detection/emphasis processing accuracyare dominant elements for image impression but the gradationquantization as the pre-stage is also important.

[0006] Furthermore, most of the recently developed mobile telephoneshave a built-in camera and can record a moving picture in real time.With such a sophisticated function, such a sophisticated application isexpected that the picked up image is made into animation in real time ora commodity is imaged and information associated with it is collectedfrom a network. In order to perform such a sophisticated imageprocessing by a small-size device, it is necessary to develop a methodcapable of performing a basic processing such as gradation quantizationat a higher speed and with a preferable final output result.

[0007] The conventional gradation quantization processing may be amethod for multi-dimensional processing in a color component (imagecomponent space) or a method for processing for each image component.The multi-dimensional processing method may be, for example, a vectorquantization method which is characterized in that the colorreproduction after quantization can be improved. On the other hand, themethod for processing for each image component may be a method forperforming analysis by repetition processing, a method for performingdivision so as to obtain a constant histogram area, a method fordividing the image component level at a constant interval, or the like.

[0008] For example, the method using the repetition processing may bethe K-average clustering method. Firstly, a reference value (level valueafter gradation quantization) is appropriately decided for each imagecomponent and while moving it with a constant width, a sum of an errorwith respect to the image component value (pixel value) is repeatedlycalculated, so as to obtain the combination of the reference values tominimize the sum.

[0009] Furthermore, as a method for performing division to obtain aconstant histogram area, there is a method to decide a reference valueso that the area of the frequency histogram of the image component valueis constant at each reference value periphery (for example,JP-A-10-164377).

[0010] Thus, there are several types of gradation quantization. For astill image, a method having a high accuracy although the processingspeed is slow has been used. For a moving picture, a method forhigh-speed processing in spite of low accuracy has been used.

[0011] The vector quantization method has a problem that processing iscomplicated, requiring a plenty of time and cannot be applied to thereal time processing of a moving picture. Moreover, the K-averageclustering method also has a problem that a plenty if time is requiredlike the multi-dimensional processing method and cannot be applied to amoving picture. Moreover, when this method is applied to the animation,there is a problem that since the reference value changes for eachframe, the color an luminance change appear as flashing of the image,causing a flicker.

[0012] Moreover, the aforementioned method disclosed in JP-A-10-164377does not consider the error between the reference value and the imagecomponent value and is valid to a certain extent when a histogram hasone peak but the quality is significantly deteriorated in other cases.Moreover, in the same way as the aforementioned method, a flicker may becaused.

[0013] The method for dividing a level at a certain interval can performa high-speed processing and since the reference value is constant forall the frames, no flicker is caused. However, this method does notanswer to the image change and the color reproduction is not preferable.This method cannot be applied to animation and obstacle extraction wherethe number of gradation should be reduced to a small number.

[0014] As has been described above, there has been no gradationquantization which can perform a high-speed processing and maintain ahigh-quality of moving picture.

SUMMARY OF THE INVENTION

[0015] An object of the present invention is to provide an imageprocessing apparatus, a mobile terminal device and an image processingcomputer readable program capable of high-speed high-quality gradationquantization processing which is required for animation of a movingpicture in real time or an object extraction by image recognitionthrough region division on a system having a low processing ability suchas a mobile telephone.

[0016] In order to realize the object, there is an aspect that an imageprocessing apparatus used comprises: an image input unit for inputtingan arbitrary moving picture, a memory device for holding the movingpicture which has been input, a processor for performing gradationquantization processing of a frame constituting the moving picture, andan image output unit for outputting the moving picture which has beensubjected to the gradation processing. And the processor modifies thereference value of the gradation quantization processing for each of theframes constituting the moving picture.

[0017] More specifically, the processor modifies the reference value ofthe gradation quantization processing by comparing a part or whole ofthe frequency histogram of the image component value associated withcolor information for the plurality of frames constituting the movingpicture.

[0018] Other objects, features and advantages of the invention willbecome apparent from the following description of the embodiments of theinvention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]FIG. 1 is a flow chart explains an image processing methodaccording to an embodiment of the present invention.

[0020]FIG. 2 is an explanatory diagram showing details of the imageprocessing method according to the embodiment of the present invention.

[0021]FIG. 3 is an explanatory diagram showing an image processingmethod according to another embodiment of the present invention.

[0022]FIG. 4 is a characteristic diagram explaining an inter-frame imageinformation comparison method used in the present invention.

[0023]FIG. 5 is a flow chart showing detailed of the image processingmethod according to the embodiment of the present invention.

[0024]FIG. 6 is an explanatory diagram explaining a gradation referencevalue calculation method used in the present invention.

[0025]FIG. 7 is a block diagram showing an image processing apparatusaccording to an embodiment of the present invention.

[0026]FIG. 8 is a block diagram showing an image processing apparatusaccording to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Description will now be directed to embodiments of the presentinvention with reference to the attached drawings.

[0028]FIG. 1 shows an image processing method according to an embodimentof the present invention and its technical features will be explainedbelow.

[0029] Firstly, in step (101), an image is input. The image may be oneimaged by a camera or a stream stored, transmitted, or broadcast. Theimage which has been input is stored in a memory for each frame. Aplurality of frames may be stored or one frame may be stored. Next,control is passed to step (102).

[0030] In step (102), image statistical processing is performed for eachframe. The image statistical processing includes a frequency histogramcreation for each pixel component value (pixel value) of RGB and YUV andcalculation of various statistical information on the image such as themaximum value, the minimum value, and the center value. This stepincludes a color component transform for image processing. Next, controlis passed to step (103).

[0031] In step (103), image characteristic amount is compared betweenframes. The image characteristic amount is a value indicating the imagecharacteristic obtained by the statistical processing in step (102). Inthis step, this image characteristic amount is compared between frames.Next, control is passed to step (104).

[0032] In step (104), a reference value for gradation quantization isadjusted. According to the comparison result determined by the precedingstep (103), a reference value or parameter for each frame is determined.The reference value or parameter for gradation quantization is the levelvalue of the image component of RGB and YUV. When the preceding step(103) decides that the characteristic amount change between frames issmall, the same reference value as the preceding frame is used and whenthe change is large, a new reference value is calculated and set. Next,control is passed to step (105).

[0033] In step (105), image processing for each frame is performed.According to the reference value decided by the preceding step (104),gradation quantization is performed. This step includes contour emphasisprocessing and smoothing in animation, region division and regioncombination in the object extraction processing, filter processing suchas shape matching, and color component transform for image display.Next, control is passed to step (106).

[0034] In step (106), an image which has been processed is output. Theoutput may be performed to a display or written/stored in a file. Thus,a sequence of processes is complete.

[0035] Next, with reference to FIG. 2, detailed explanation will begiven on the embodiment of the present invention. Explanation will begiven on a method as an example in which an imaged moving picture isinput and each frame is made into animation.

[0036] In step (101), an image is input and control is passed to step(201).

[0037] In step (201), color component transform is performed. Forexample, an image picked up by a CCD (charge coupled device) has RGBimage components as a color component. Here, as an example, a colorcomponent is transformed from RGB to YUV. A distance between two pointsin the YUV space is nearer to the color distance felt by human sightthan in the RGB space. For this, by performing processing in the YUVspace, it is possible to reduce a color error generated by quantizationand viewed by a human. There is also a CIE-L*a*b* space considering thehuman psychological color distance. However, this complicates thetransform processing and here, the YUV space is used. The transformmethod may be an ordinary method. The color component transform may notbe performed. Next, control is passed to step (202).

[0038] In step (202), a frequency histogram is created for each imagecomponent. The frequency histogram is used as the image characteristicamount in FIG. 1. Since transform into the YUV space has been performedin the aforementioned step (201), a frequency histogram of a currentframe is created for each of YUV and stored in memory. The frequencyhistogram may be created either for all of the YUV or some of them, forexample, only for Y. When the histogram is created for all of the YUV,in the later step, it is possible to accurately judge the image changewhen performing comparison between frames. On the other hand, when thehistogram is created only for Y, it may not possible to grasp when onlya color is changed between frames such as illumination condition withinthe image. However, it is possible to grasp image change accompanyingluminance change such as object movement and save memory and performprocessing at a high speed because no other histograms are created.Next, control is passed to step (203). Step (201) and step (203)correspond to step (102).

[0039] In step (203), the frequency histograms of the current frame andthe preceding frame are compared. In this step, by comparing informationon the image which has been subjected to statistical processing, it isjudged how the current frame of the moving picture has changed ascompared to the preceding frame. When a larger change is present, thereference value is re-calculated. Otherwise, no re-calculation isperformed and the same reference value as the preceding frame is used.Comparison may be performed for all of the YUV or only for Y. Thehistograms of the both frames are superimposed and are compared tocalculate the area of the different parts of two histograms to determinetheir percentage of the area of the histogram (i.e., total number ofpixels of one frame). For example, when the percentage exceeds 30%, itus judged to perform re-calculation of quantization reference value, andwhen the ratio is below 30%, the same reference value as the precedingframe is used. When comparison is performed for a plurality ofcomponents, re-calculation may be performed if all the components exceeda certain value or if one of the components exceeds a certain value.When a plenty of noise is present because of the image condition used,the processing becomes stable when judgment is performed when all thecomponents exceed a certain value. When noise is small and it isnecessary to be sensitive to a color and luminance change, judgment isperformed when one of the components exceeds a certain value. Thecertain value as a judgment condition is predetermined in advance. Thevalue may be dynamically changed depending on the condition. It is alsopossible to provide an interface so that a user can set as is necessary.

[0040] Explanation will be given on the method of calculating the ratioof the area of the different parts of the histogram with reference toFIG. 4. FIG. 4 shows a frequency histogram h(l) concerning an imagecomponent value 1, of luminance and chroma. A graph (401) represents ahistogram of the current frame and a graph (402) represents a histogramof the preceding frame. When the luminance value Y is to be compared, ifthe Y range of the input image is 0 to 255, absolute values ofdifferences of the histograms of the both frames in this range aretotaled and divided by the total number of pixels of one frame. Thisstep corresponds to step (103). Here, the histogram area is compared. Itis also possible to determine whether the peak values of the histogramsare at a distance exceeding a certain value or to compare the frequencyupper points to determine whether they are identical so as to obtain acharacteristic amount between the frames. Instead of simple histogramcomparison, it is also possible to compare histograms multiplied by anerror coefficient which will be detailed later. Moreover, it is alsopossible to determine a reference value according to a reference valuedetermination method which will be detailed later and stores it so thatreference values are compared for judgment. Thus, by performingcomparison between image statistical information instead of comparingthe images themselves, it is possible to reduce the memory required forstoring data as compared when storing the image. Next, control is passedto step (204) or step (205) according to the judgment.

[0041] In step (204), the reference value is set to the same referencevalue of the preceding frame. Next, control is passed to step (206).

[0042] In step (205), re-calculation of the quantization reference valueis performed. As the reference value calculation method, an error peaktrace method which will be detailed later is used in this embodiment.Next, control is passed to step (206). Step (204) and step (205)correspond to step (104).

[0043] In step (206), by using the reference value set in the precedingstep, gradation quantization of each frame is performed. The gradationquantization is matched with the nearest reference value in which thelevel value is set for each pixel component of all the pixels in theimage, i.e., YUV here. For example, when the Y reference value is (0,128, 255), the U reference value is (−50, 0, 50), and the V referencevalue is (−30, 10, 40), the pixel of [Y, U, V]=[10, 15, 15] is quantizedinto [0, 0, 10]. Next, control is passed to step (207).

[0044] In step (207), filter processing is performed. The filterprocessing is image processing including contour emphasis processing andsmoothing in animation processing, region division and regioncombination in an object extraction processing, shape matching and thelike. For example, in the contour emphasis processing in the animation,the edge emphasis processing used in the ordinary image processing maybe performed. In this invention, the UV component of each pixel isplotted as a vector from the origin onto the UV 2-dimensional plane.When the angle defined by the vectors of the target pixel and aperipheral pixel is above a predetermined angle, it is judged that acontour is present at the pixel boundary of them and the Y componentvalue is divided by a predetermined value so as to display the contourconcentrated and emphasize it. As for the smoothing, the medianfiltering is used for the Y component and averaging of eight vicinitypixels is performed for the UV component. Moreover, it is also possibleto apply the ordinary filter processing such as the none-sharp maskingand noise removal. Next, control is passed to step (208).

[0045] In step (208), color component transform is performed. Here, forpixel display, the color component is returned from the YUV to RGB.Next, control is passed to step (106). Steps (206) (207) (208)correspond to step (105).

[0046] In step (106), an image is output and a series of operation iscomplete.

[0047] Description will now be directed to another embodiment of thepresent invention with reference to FIG. 3. This embodiment ischaracterized in that the Y component of the YUV space is processed byother method.

[0048] After an image is input in step (101), control is passed to step(201) and then to step (202). Steps (201) and (202) correspond to theaforementioned step. Next, for the Y component among the YUV imagecomponents, control is passed to step (303) and for the UY components,control is passed to step (203).

[0049] In step (303), a reference value is set at an identical intervalin the Y component range. For example, when the Y component range is 0to 255 and five reference values are decided, they are set to (0, 64,128, 192, 255). Here, the Y component alone is handled separatelybecause the human sense of sight differs between the luminance componentand the chroma component and more sensitive to the luminance change thanto the chroma change. Depending on an application used, there is a casewhen it is better to handle the luminance and the chroma by the samemethod and there is a case when it is better to handle them separately.For example, when the reference value is set at an identical intervalfor the luminance, if the number of the reference values is small, muchnoise is present in the entire image as compared to the error peak tracemethod which will be detailed later, but the edge in a small portion caneasily be stored. Next, control is passed to step (206).

[0050] Step (203) performs the processing similar to the aforementionedprocessing for the UV components. After this, steps (204), (205), (206),(207), (208), and (106) are performed in the same way as has beendescribed above.

[0051] Next, referring to FIG. 5, detailed explanation will be given onthe method for calculating the quantization reference value according tothe present invention. This correspond to step (205) and is called theerror peak trace method.

[0052] When calculating a quantization reference value, firstly, in step(501), an error coefficient fi(l) is calculated. The fi(l) is an errorcoefficient concerning the image component value 1 of the luminance andchroma for calculating the i-th reference value. For example, as theerror coefficient fi(l), Equation 1 is used. $\begin{matrix}{\left\lbrack {{Equation}\quad 1} \right\rbrack {{f_{i}(l)} = \left\{ \begin{matrix}{{1\quad \ldots \quad i} = 1} \\{{\min\limits_{l,{j < i}}{{\left\lbrack {{l - s_{j}}} \right\rbrack/L}\quad \ldots \quad i}} \neq 1}\end{matrix} \right.}} & (1)\end{matrix}$

[0053] Here, Si is the i-th reference value, j is a variable indicatingthe number of reference value smaller than i, and L is a range width ofthe image component 1. The range width indicates a width each componentvalue may have. In the Equation 1, when calculating the first referencevalue, f1(l)=1 and the error coefficient is constant for all the 1 andwhen calculating the second reference value and after, for all the 1,the nearest distance with respect to the reference values acquired isdivided by the range width to obtain the error coefficient. Here, theerror coefficient is as shown in Equation 1. It is also possible tomultiply this by a constant coefficient so as to be proportional tothis. That is, it is also possible to use a value proportional to thedistance to the nearest reference value from each image component value.Next, control is passed to step (502).

[0054] In step (502), by using the histogram h(l) and the errorcoefficient fi(l) obtained in the preceding step (501), for all thecomponent 1, the products fi(l)×h(l) are calculated and the 1 having themaximum value is made the i-th reference value Si. The product of thehistogram and the error coefficient is a value indicating the error whengradation quantization is performed by using the reference valuesacquired. By selecting its peak, it is possible to acquire an imagecomponent value having the maximum error. This can be expressed byEquation 2 as follows.

[0055] [Equation 2]

S _(i) ={l|[h(l)×f _(i)(l)]}  (2)

[0056] When there are more than one maximum values, the an arbitraryvalue such as median is selected. Next, control is passed to step (503).

[0057] In step (503), the number N of the reference values to beacquired is compared to the number i of the reference values which havebeen acquired. When all the reference values have been acquired, theprocessing is terminated. When not all the reference values areacquired, the number i is incremented by one and control is returned tostep (501). Here, judgment is made according to the number of thereference values to obtain the end condition. However, it is alsopossible to judge whether the product fi(l)×h(l) of the histogram h(l)and the error coefficient fi(l) exceeds a predetermined value, so thatwhen a part exceeds the predetermined value control is returned to step(501) to continue acquisition of the reference values and otherwise, theprocessing is terminated. Moreover, it is also possible to total theproducts fi(l)×h(l) of the histogram h(l) and the error coefficientfi(l) all the 1 and judgment is made whether the total exceeds apredetermined value, so that when a part exceeds the predeterminedvalue, control is returned to step (501) to continue the acquisition ofthe reference value and otherwise, the processing is terminated. Sincethe product of the histogram and the error coefficient indicates anerror amount caused by the gradation quantization in the image, whenthese methods are used, it is possible to adjust the image quality afterquantization depending on the predetermined value.

[0058] Next, referring to FIG. 6, detailed explanation will be given onthe error peak trace method.

[0059]FIG. 6 explains the method for calculating the first to thirdreference values and error coefficients. Firstly, when calculating thefirst reference value, i.e., i=1, f1(l)=1. That is, f1(l)×h(l) is equalto h(l) and the calculation of the peak of this according to Equation 2is identical to the calculation of the level value which is the peak ofthe histogram. This is set as the first reference value. Next, thesecond error coefficient is calculated from Equation 1. Since Equation 1uses as an error the distance from the nearest reference value among thereference values which have been selected, the error coefficient off2(l) makes a V-shaped graph based on the reference value Si as shown inthe figure. This is multiplied by the histogram value according toEquation 2 so as to obtain the image component value of the maximumerror and the peak is traced to select the second reference value S2.The third reference value and after are obtained in the same method.

[0060] As another method, the first reference value may not be thehistogram peak but the image component range center or the bisectingpoint value which is the level value at the position bisecting thefrequency histogram area and Equation 1 may be used for i ≧2. Since theerror peak trace method uses different reference values selectedaccording to the setting of the initial value, by varying the initialvalue according to the condition, it is possible to adjust the imagequality according to the application used.

[0061] Moreover, instead of using the frequency histogram simply ash(l), it is possible to use a weighted frequency histogram created byincreasing the weight toward pixel at the screen center when calculatingthe frequency. By using this method, it is possible to increase thecolor reproduction of an object at the center of the screen after thegradation quantization as compared to the periphery.

[0062] Next, referring to FIG. 7, an explanation will be given on animage processing apparatus according to an embodiment of the presentinvention.

[0063] The image processing apparatus according to the present inventionincludes an image input unit (701), a memory device (713), a processordevice (714), and an image output unit (702).

[0064] The image input unit (701) is means for inputting a movingpicture. The moving picture may be an image picked up by a camera or astream stored, transmitted, or broadcast. The images which have beeninput are stored in a frame memory (703) in a memory device (713) foreach frame.

[0065] The image output unit (702) is means for outputting a movingpicture. The output may be display on a display or writing/storage intoa file.

[0066] The memory device (713) is a memory for storing a frame image andinformation associated with this. The function of the memory device(713) is divided into a frame memory (703), a histogram memory (704), acomparison histogram memory (705), a reference value memory (706), and afilter processing memory (707).

[0067] The frame memory (703) stores a frame image for processing. Itcan store a plurality of frames or a single frame. The histogram memory(704) is a memory for creating and storing a histogram associated witheach image component of the current frame. The comparison histogrammemory (705) is a memory for storing a histogram of the preceding framefor comparison. The reference value memory (706) is a memory for storinga reference value for gradation quantization determined by the referencevalue decision method. The filter processing memory (707) is a memoryused for the filter processing.

[0068] The processor device (714) is a processor for performing theimage processing and the statistical calculation. The function of theprocessor device (714) is divided into a color transform unit (708), ahistogram processing unit (709), a reference value calculation unit(710), a quantization unit (711), and a filter processing unit (712).

[0069] The color transform unit (708) modifies the image componentassociated with color information from RGB to YUV or vice versa. Thecolor component transform may be performed by using an ordinary method.The histogram processing unit (709) checks pixels of the frame image,creates a frequency histogram for each image component, and stores it inthe histogram memory (704) and in the comparison histogram memory (705).The reference value calculation unit (710) calculates a reference valueaccording to the reference value decision method and stores it in thereference value memory (706). The quantization unit (711) quantizes theimage according to the gradation quantization method. The filterprocessing unit (712) performs the filter processing to the frame image.

[0070] In the image processing apparatus of FIG. 7, means for therespective functions are described for explanation. However, these meansmay be mounted together on a computer chip or realized by software.

[0071]FIG. 8 shows the processing method of the present inventionapplied to a mobile telephone having a camera. The image input unit(701) corresponds to the camera and the display (803) and the storagedevice (804) correspond to the image output unit (702). According to auser instruction from a user interface (801), the image input from thecamera is subjected to the gradation quantization and filter processingaccording to the aforementioned method, displayed on a display screen(803) and store din a storage device (804). The moving picture which hasbeen processed may be attached to a mail to be transmitted from a radiocommunicator (802). It is also possible to streaming-transmit theprocessed moving picture in real time from the radio communicator (802).The aforementioned gradation quantization method is not heavy as aprocessing not requiring CPU processing ability and can be operated in asmall-size terminal such as a mobile telephone.

[0072] It should be noted that the aforementioned embodiment has beenexplained through real time animation and region division in an obstacleextraction but the present invention is not limited to this.

[0073] As has been explained above, in the field of image recognition,what has become important is the technique of object extraction forisolating an object such as a person and a vehicle from the image pickedup and the technique for preparing animation from the moving picture.

[0074] The object extraction processing is realized by performing thegradation quantization processing, region dividing processing, shapematching, and the like in cooperation. The region division is aprocessing for dividing an image which has been subjected to colorreduction by the gradation quantization, into a plurality of regions bygrouping areas having similar colors. The shape matching is a processingfor comparing each of the regions obtained by the region division toshape information held as data, thereby unifying the regions to verifywhether a particular object is present. Thus, an image processing isrealized by combining various processes. Among them, the basicprocessing such as gradation quantization is important.

[0075] Recently, there is software for acquiring a moving picture pickedup by a video camera into a PC and editing it. By adding a simple effectto the moving picture picked up, it is possible to create a new movingpicture. In future, an animation method is expected and as a pre-stageof it, realization of gradation quantization has become important.

[0076] By employing the present invention thus far explained to thegradation quantization, it is possible to realize gradation quantizationcapable of high-speed high-quality output. The present invention can beapplied to an image processing apparatus such as the object extractionprocessing and animation so as to improve the performance of thesedevices and effectively operate them.

[0077] According to the present invention, in the gradation quantizationprocessing when performing the animation of a moving picture or a regiondivision in an object extraction, by comparing inter-frame histogramcomparison, it is possible to suppress generation of flicker in thereproduced image. Moreover, by using the error peak trace method forsuccessively calculating the quantization reference value according tothe product of the frequency histogram and the error coefficient, it ispossible to realize a gradation quantization processing capable ofhigh-speed processing and obtaining a high-quality output image. Thus,it is possible to mount the moving picture animation function and theobject recognition function on a small-size device such as a mobiletelephone.

[0078] It should be further understood by those skilled in the art thatalthough the foregoing description has been made on embodiments of theinvention, the invention is not limited thereto and various changes andmodifications may be made without departing from the spirit of theinvention and the scope of the appended claims.

What is claimed is:
 1. An image processing apparatus comprising: animage input unit for inputting an arbitrary moving picture, a memorydevice for holding the moving picture which has been input; a processorfor performing a gradation quantization processing of a frameconstituting the moving picture, and an image output unit for outputtingthe moving picture which has been subjected to a gradation processing,wherein the processor modifies the reference value of the gradationquantization processing for each of the frames constituting the movingpicture.
 2. An image processing apparatus as claimed in claim 1, whereinthe processor modifies the reference value of the gradation quantizationprocessing by comparing a part or whole of the frequency histogram ofthe image component value associated with color information for theplurality of frames constituting the moving picture.
 3. An imageprocessing apparatus as claimed in claim 1, wherein the processor maynot modify the reference value according to the frames.
 4. An imageprocessing apparatus as claimed in claim 2, wherein the processor maynot modify the reference value according to the frames.
 5. A computerreadable program causing a computer to execute an image processingmethod comprising the steps of: inputting a moving picture, performingstatistical processing associated with pixel values for each of theframes constituting the moving picture, comparing image characteristicamounts which have been subjected to the statistical processing betweenthe frames, resetting the reference value for each of the framesaccording to the comparison result, and performing a gradationquantization processing of the frames according to the reference value.6. A computer readable program as claimed in claim 5, wherein the stepof setting the reference value may not reset the reference valuedepending on the comparison result.
 7. A computer readable program asclaimed in claim 5, wherein the step of performing the statisticalprocessing associated with the pixel values creates a frequencyhistogram for each of the frames, and the step of performing acomparison compares a part or whole of the histogram.
 8. A computerreadable program as claimed in claim 6, wherein the step of performingthe statistical processing associated with the pixel values creates afrequency histogram for each of the frames, and the step of performingcomparison compares a part or whole of the histogram.
 9. A computerreadable program as claimed in claim 5, wherein the step of resettingthe reference value calculates the reference value by tracing the peakof the product of the histogram and the error coefficient.
 10. Acomputer readable program as claimed in claim 6, wherein the step ofresetting the reference value calculates the reference value by tracingthe peak of the product of the histogram and the error coefficient. 11.A computer readable program as claimed in claim 7, wherein the step ofresetting the reference value calculates the reference value by tracingthe peak of the product of the histogram and the error coefficient. 12.A computer readable program as claimed in claim 9, wherein the errorcoefficient used when calculating the i-th reference value is a valueproportional to an absolute value of a difference between each pixelvalue and the nearest-to-the-pixel-value reference value among thereference values up to the (i−1)-th reference value.
 13. A computerreadable program as claimed in claim 10, wherein the error coefficientused when calculating the i-th reference value is a value proportionalto an absolute value of a difference between each pixel value and thenearest-to-the-pixel-value reference value among the reference values upto the (i−1)-th reference value.
 14. A computer readable program asclaimed in claim 11, wherein the error coefficient used when calculatingthe i-th reference value is a value proportional to an absolute value ofa difference between each pixel value and the nearest-to-the-pixel-valuereference value among the reference values up to the (i−1)-th referencevalue.
 15. A mobile terminal device comprising a camera, a control unit,and a display unit, wherein the control unit includes a computerreadable program having a step of inputting a moving picture, a step ofperforming a statistical processing associated with a pixel value foreach of frames constituting the moving picture, a step of comparingimage characteristic amounts subjected to the statistical processingbetween the frames, a step of resetting the reference value for each ofthe frames according to the comparison result, and a step of performinggradation quantization processing of the frames according to thereference value, and wherein the moving picture picked up by the camerais processed and displayed on the display unit.